Message queues are software components that provide an asynchronous communications protocol for applications and interprocess or inter-thread communication. Message queues provide an asynchronous communications protocol in that the sender and receiver do not need to interact with the message queue at the same time. In a typical message queue implementation, a system administrator installs and configures a commercially available message queue manager and defines a named message queue. An application registers a software routine that listens for messages placed in the queue, and other applications may then connect to the queue and transfer messages onto it. The queue manager stores the messages until a receiving application connects and calls the registered software routine. The message can then be processed by the receiving application.
The efficient processing of messages is critical in enterprise software systems and cloud computing platforms that utilize content servers for electronic record management and management of real-time activities and information, such as in business activity monitoring (BAM) and business process management (BPM) solutions. In most current systems, a single message queue, or message service, such as Java Message service is created and used by different applications. This creates a significant message processing bottleneck and limits the flexibility of message processing.
There are several well-established use cases of message queues in a content server, such as a workflow task queue, fulltext index task queue, and business process execution monitoring. For the workflow task queue, a queue table is created to support workflow/router functionality. As a workflow executes, some tasks become ready for execution and are entered to this queue. The designated performer of the task acquires and executes the task, and brings the workflow execution to the next state. For fulltext indexing, the content server does full-text indexing asynchronously. When an object is created, updated, or destroyed, the content server generates a fulltext indexing request and adds the request to a queue. The fulltext index agent retrieves the request from the queue and calls the indexing server to update the fulltext indexes accordingly. In the BAM context, to monitor business process execution status, various events about business process activities must be recorded and then extracted by the BAM server into an integration table for further format and aggregation processing. Each of these use cases, among others, present certain issues with regard to current single message queue systems. In general, sharing a message queue by multiple features presents performance disadvantages. The workflow task queue is a hot spot in the system because of frequent add/delete to/from actions in the queue. For example, a workflow task execution causes frequent updates to set the task state, and updates to individual tasks are done on a one-by-one basis. The same is true for handling fulltext indexing tasks. As the task queue size grows, queries and updates on the queue will slow down because of various indexes defined over the queue table. This is especially true in some special cases, such as when many fulltext indexing requests are generated, while the index agent is not able to consume at a consistent rate to keep the queue at a manageable size.
Another issue with the existing queuing mechanisms is that the message schema is generally fixed. That is, message schemas are specifically defined for specific workflows and cannot be used for other purposes in general. Present message queue systems also suffer from transactional limitations in that they use polling to find available index requests, which presents an intrinsic problem of controlling the polling interval. Frequent polling causes unnecessary system resource consumption while overly long polling intervals result in unacceptable latency.
The issues associated with present message queue systems are generally caused by the content servers not providing a general message queue mechanism that different applications can use to create and manage their own queues. Although the Java Messaging System may provide the possibility of using message services provided by third-party providers, using such external message services may not meet the performance requirements of the use cases in content server environment as listed above. In addition, using JMS introduces additional deployment and administration complexity. For example, transaction management can also be problematic in that two phase commit (2PC) protocols may be needed when using third party message services.
What is needed, therefore, is a generic and flexible message queue mechanism that allows applications to create and manage their own message queues. What is further needed is a message queue system that eliminates unnecessary polling by providing a notification mechanism such that a message consumer will be blocked when there are no pending messages available in the queue, and the blocked message consumer is notified as soon as a message becomes available.